edam 



ISSN 1303-0485 • elSSN 2148-7561 
DOI 10.12738/estp.2015.2.2426 

Copyright© 2015 EDAM • http://www.estp.com.tr 

Educational Sciences: Theory & Practice • 2015 April • 15(2] • 359-376 


Received I 21 January 2014 
Accepted I 25 December 2014 
OnlineFirst I 30 April 2015 


Monitoring of Educational Performance Indicators in 
Higher Education: A Comparison of Perceptions 


Hiiner §encan a 

Istanbul Commerce University 

A. Tugba Karabulut b 

Istanbul Commerce University 


Abstract 

The purpose of this study is to explore whether there is a statistically significant difference between the ideas of 
university administrators and faculty members regarding how strictly Educational Performance Indicators for 
Educators (EPIE) should be monitored in the educational process. The responses of university directors were 
compared with those of faculty members, and the responses of public universities were compared to those of 
private universities. Improving the quality of education depends on the performance of teachers. Determining 
the objective indicators that measure the performance levels of teachers is necessary for maintaining quality 
assurance. In order for a university to maintain high standards, administrators and faculty members must 
cooperate with each other. The twelve point EPIE Index used in this study contains three components and nine 
indicators. No statistically significant relationship was found between the average component scores of uni¬ 
versity administrators and faculty members, and no statistically significant relationship was found between the 
average component scores of public and private universities. However, a statistically significant relationship was 
found between two factors from EPIE’s self-appraisal scores. The model and data do not match the conceptual 
structure of the three components in the EPIE. Thus, further studies and analyses should be conducted. It was 
found that most faculty members think like academicians from private universities, and most university direc¬ 
tors think like academicians from public universities. 
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Monitoring of educational performance indicators 
for educators (EPIE) has gained importance for 
those interested in designing education processes 
appropriately, improving education, and forming 
these processes creatively and innovatively. It is 
not possible for department chairs and deans to 
form and monitor EPIE alone. Instead, they need 
to cooperate and work with the other faculty 
members. Without this, data related to EPIE could 
not be collected or analyzed, and preparing and 
submitting meaningful knowledge to university 
directors would be impossible. This situation is 
encountered when EPIE is determined solely by 
university directors. University directors have a 
problem determining how to collect and appraise 
EPIE from their faculty members. What level 
should be monitored in collecting data from 
EPIE; how should it be analyzed and reported? 
These are all topics of recent discussion. Faculty 
members want to observe the benefits of collecting 
data for EPIE. Increasing the preparation, 
learning, and satisfaction levels of students; the 
quality and sensitivity in the education process; 
the transformation of faculties from the role of 
knowledge transmitter to the role of initiator of 
thought; research; and appraisal; these are all 
convincing factors for faculty members. This study 
compares the perceptions and ideas of university 
professors with the directors in order to determine 
at what level EPIE should be monitored. It was 
assumed that university directors would be more 
sensitive and have higher expectations about EPIE. 
One finds that organizations cause similar findings 
for the expectations of university directors and 
faculty members. At what level does the sensitivity 
of the EPIE for these groups bring them together? 
If it is at low levels, it will not only render EPIE 
useless, but will also put the focus on protecting 
internal balance instead of developing innovative 
and creative processes. Forming and monitoring 
EPIE with collaboration in higher education are 
important for protecting and improving the quality 
of education. 

Literature Review 

Developmental Process of EPIE: Observation of 
EPIE started at the end of the 1980 s and beginning 
of the 1990 s. During this time, performance 
reporting was considered as a main tool for policy 
developers and for providing information to the 
public (Cabrera, Colbeck, & Terenzini, 2001), (Shin, 
2010). The British government declared EPIE to be 
a requirement in higher education with The Green 


Paper issued in 1985 (Ball & Hallwachi, 1987). The 
University Management Statistics and Performance 
Indicators Report was published in the UK in 1996 
based on this requirement (Cave, Hanney, Henkel, 
& Kogan, 1997). Although the UK has since started 
to use the term Quality Statistics in Higher Education 
in place of Performance Indicators, most countries 
continue to use the latter. Researchers separated the 
terms “performance” and “intention,” considering 
indicators as a sign instead of as an absolute appraisal 
unit (Sizer, Spee, & Bormans, 1992). Indicators are 
considered guide values for understanding whether 
determined goals are reached or not. However, the 
statistics should be reliable and valid. Performance 
indicators for the second group were related to 
education and learning when they were applied in 
the UK for the first time. This group focused on such 
values as {a) the percentage of students who lengthen 
their education and ( h) the percentage of students 
who dropped out of their school (Draper & Gittoes, 
2004). The number of performance indicators has 
increased and attracted the attention of researchers 
in education and learning. 

Determining EPIE: Research has been conducted 
on performance indicators related to the education 
process in the class environment in addition 
to general performance indicators in higher 
education. According to Cabrera et al. (2001), 
universities focused on the performance indicators 
which showed financial stability in the past. Since 
traditional indicators do not show student success 
levels, performance indicators related to class 
education need to be established. In establishing 
these, attention needs to be given to how, by whom, 
and at which level these performance indicators 
will be determined. The current researchers were 
unable to find any study showing the ideas of 
university directors and faculty members regarding 
the level to monitor and appraise EPIE. Lei (2013) 
evaluated the monitoring concepts: innovative 
learning, self-learning, education atmosphere, 
educational interaction, educational guidance and 
inspiration, the selection of education materials, 
defining hard tasks, practicing new education 
concepts, and innovation in education methods. 
He, divided EPIEs into six categories: education 
goals, education behaviors, education atmosphere, 
learning attitude, education characteristics and 
innovation, and self-developmental ability of 
students. He organized these six indicators in 
four dimensions and analyzed them with fuzzy 
logic theory. Fernandez et al. (2011) defined 
organizational performance indicators as 
“objectively appraisable corporate objectives.” They 
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tried to appraise performance indicators using the 
Bayesian network method even though it had no 
statistical base. Popa (2011) conducted a study on 
the importance of performance indicators which 
are mostly qualitative yet sometimes quantitative, 
and she found that they are very important tools for 
appraising education activities. According to Popa, 
the Council for Higher Education Accreditation 
defines EPIEs as the outcomes and findings 
of the education program and education process. 
Indicators are tangible indicators used to make 
announcements to the public in a socially 
responsible and transparent manner.” While Sousa 
and Veiga (2009) suggested that indicators can be 
quantitative, the current researchers believe that 
qualitative data is more reliable. 

Cuenin, the pioneer of EPIE studies, established the 
minimum condition for performance indicators 
as “digital values that may be used for assessing 
the quantitative success of a system” (Ball & 
Wilkinson, 1994). It is understood that Popa gives 
particular importance to performance indicators 
in her study. The approach used in this study, 
however, concentrates solely on the establishment 
of the success indicators that belong to academicians 
through the education process. As an initial study, 
ascertainment of administrator and academician 
opinions regarding the correct level of how to 
monitor the determined EPIEs was researched. 
Under this context, the education process starts 
when the academician prepares a single semester 
course program, proceeds with timely presence in the 
class, teaches the course efficiently, measures student 
learning, and ends with satisfied students. The other 
performance indicators that affect education quality 
globally require organization variables rather than 
just the educator s performance. These are not within 
the scope of the current study. 

Educational Performance Indicators Covered 
under Input, Process and Consequences: 

Education performance indicators may be divided 
into three groups: beginning, process, and results. 
The beginning defines the input indicators, among 
which are educational expenses per student, general 
indoor area per student, classroom area per student, 
student-teacher ratio, number of students per student 
affairs officer, number of students per computer, 
social activity expenses per student, and so on. Input 
performance indicators are related to the educational 
infrastructure of the organization and define the 
financial, physical, and human resources necessary 
to provide quality training. Popa used a model 
developed by Kirkpatrick in assessing the educational 


process. This model foresees at least four levels for 
assessing education with each level including at least 
one indicator. The four levels from Kirkpatrick are 
satisfaction with the education received, with the level 
of acquiring knowledge, with the level of knowledge 
reflected in the behavior of the student, and with the 
organizational results. The final level decides whether 
the program is successful or not (Popa, 2011). 
Some performance indicators related to the results 
of education are timely graduation rates, dropout 
rates, scholarship acquisition rates due to success, 
employment rate within six months of graduation, 
GPA percentages higher than 3.0 or 3.30 (based on a 
4.0 grade scale), the percentage of graduate students 
who take the ALES exam (the test required to enter 
graduate school) and score higher than 70%, the rate 
of students who enter the foreign language exam 
(YDS) and score at least 65%, and the rate of the 
students continuing on to graduate studies. 

As is seen, existing performance indicators linked 
to the results show whether the four year education 
process has been successful or not. However, 
these indicators are related to the completed stage 
of the process. In order for such indicators to 
obtain good standing, one should determine and 
improve certain performance indicators. Thus, 
researchers have concentrated on such titles as 
“the frequency of weekly library use by students,” 
“how often students participate in group activities,” 
“how readily leadership positions are assumed,” 
“homework completion levels,” “whether students 
solicit and receive opinions from their advisors 
on their research topics, courses, and so forth” 
(Gaither, Nedwek, & Neal, 1994). The purpose 
of the current study is to focus on the education 
process, leaving the input and output performance 
indicators in the hands of other researchers. 

Performance Indicators Related to the Education 
Process: Performance indicators within the 
education process may be handled in three groups: 
(a) performance indicators focusing on students, ( b ) 
performance indicators focusing on academicians’ 
teaching responsibilities, and (c) performance 
indicators focusing on the administration. An 
important portion of educational activities is related 
to the organizational and administrative processes, 
not to the teaching-learning process. For instance, 
the number of students per academician, class size, 
frequency of technology use in education, number 
of new enrollments, how beneficial education 
programs are, and the proportion of academicians 
conducting scientific research and related activities 
are some of these types of indicators (Serdar, 2010). 
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Since the education process must be considered 
as a whole, the effect that the administration 
has on education should not be underestimated. 
With this being said, however, only academicians’ 
educational responsibilities were focused on in 
this study. The main functions of vice-deans and 
division heads responsible for academic affairs is 
to improve educational processes directly related 
to academicians and assist them in overcoming the 
difficulties they face when making such attempts. 

Karsten, Visscher, Dijkstra, and Veenstra (2010) 
claim that although the educational performance 
indicators that bring prestige to an organization 
show the relative superiority of that organization, 
they may not provide an idea as to whether 
absolute educational standards have been fully 
met or not. Since absolute educational standards 
do not have a generally accepted definition, higher 
education institutions compare their positions 
over a specific period of time or against the values 
of another institution that they accept as superior. 
In doing this, they try to place themselves in a 
specific position. Karsten et al. (2010) claim that 
education performance indicators should meet 
“strict methodological parameters,” in other 
words, the validity, reliability, and integrity of the 
parameters. An additional parameter, “usability,” 
may be included. Some indicators do not have 
visible or understandable results, and others may 
even demonstrate rather uncertain results. Both 
academic directors and academicians should 
carefully determine and monitor the performance 
indicators which clearly generate “added value” 
and have a positive effect on student development. 
Twelve theoretical educational-performance 
indicators in this study have been selected and 
named Educational Performance Indicators for 
Educators (EPIE). The basic characteristic of 
performance indicators is to provide information 
as to the how frequently academicians realize their 
initial targets. For the EPIE system to function 
properly, it must be meaningful for academicians, 
useful for the organization, and include easy data 
compilation methods and means of reporting. 
Several studies have been performed aiming to 
determine performance indicators. For instance, 
Chen et al. (2009) gathered data from higher 
education institutions using the Delfi method, 
determining 78 performance indicators falling 
under 18 dimensions. They then attempted to 
generate an organizational balanced scorecard for 
higher education institutions depending on such 
performance indicators (Shun-Hsing, Hui-Hua, & 
King-Jang, 2009). The study of Shun-Hsing et al. 
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covers the entirety of the organization. However, 
determining those performance indicators which 
focus only on the education process and those 
which increase academician engagement in the 
actual educational process are more important for 
department heads. 

The twelve theoretical educational-performance 
indicators as determined by the researchers are 
meaningful for universities in Turkey that want 
to reach their educational goals. The important 
thing is not to increase the number of performance 
indicators but to generate a change in behavior by 
using fewer indicators so as to increase academician 
effectiveness and efficiency, maintain student 
satisfaction, and assist in the development of the 
education process. 

Consensus between Administration and 
Academicians on Performance Indicators and 
their Level of Implementation: There is a need 
for administrators and academicians to reach a 
consensus as to what performance indicators should 
be and what degree they should be implemented 
and monitored. If such a consensus can be reached, 
the process will become part of the organizational 
culture and gain permanency. Otherwise, even 
if the most useful performance indicators are 
determined, they may not be fully implemented or 
monitored. The level of importance for performance 
indicators differs for university chancellors, faculty 
deans, and department heads. Although chancellors 
and deans give a great degree of importance to 
educational and learning activities, they either 
don’t demonstrate the same level of sensitivity to 
the details of educational and learning indicators or 
they don’t allot time to the data and tables presented 
to them. Department heads, on the other hand, try 
to increase the number of educational indicators. 
Since educational performance indicators are 
related to the success of the department. Further, 
educational performance indicators reflect on other 
indicators in one way or another. It is important 
that department heads perform joint activities 
with academicians and determine together which 
indicators should be used and to what degree they 
should be implemented and monitored. 

Question and Goal 

The question addressed by the study is bi-directional. 
First, what EPIE should be and to what degree it 
should be implemented and monitored. For this, 
first aspect, there exist differences (/) between public 
and private universities and (it) between directors 
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and academicians. The second aspect is the fact 
that individuals may think differently as to the 
extent that EPIE should be monitored depending 
on self-assessment types. The lack of consensus 
on the extent that EPIE should be monitored then 
prevents them from being effectively implemented. 
The very existence of educational performance 
indicators for educators may even be questioned in 
Turkish universities. Global performance indicators 
currently being applied by certain universities only 
have a limited effect on improving the learning 
process. EPIE should be determined and monitored 
at a specific level so as to enable universities to 
compare the quality of education they offer. The goal 
of this study is two-fold. The first is to compile the 
opinions of university directors and academicians as 
to what extent that EPIE should be monitored, and 
the second is to establish relations between the EPIE 
scores according to certain demographic groups 
such as private universities, public universities, and 
self-assessment groups. 

Method 

Design 

This study is basically descriptive, but also may 
be defined as inferential research, which includes 
the testing of certain hypotheses. Educational 
Performance Indicators for Educators, which 
consists of twelve indicators, was selected as the 
basic conceptual structure and handled theoretically 
under five dimensions: ( a ) monitoring the teaching 
course, ( b ) monitoring the use of course materials, 

(c) monitoring the course success of the students, 

( d ) monitoring the attendance of the students 
and (e) monitoring the course satisfaction of the 
students. 

No study based on the classification of performance 
indicators using a scale-index could be found in the 
literature. The aforementioned sub-dimensions 
may differ according to the specific branch of 
science being studied (whether the university’s 
focus is on research or educational activities) and 
the size of the university (big or small). After going 
through the literature, it was found that the five 
above-mentioned dimensions had covered the 
academician based education process. The model of 
the study began by examining the relations between 
(a) public and private universities and their EPIE 
component average scores, ( b ) administrators and 
academicians and their EPIE component average 
scores, and (c) self-assessment groups and their 
EPIE component average scores. 


Population and Sample 

Although the current researchers had aimed 
to collect the study’s data from universities in 
Istanbul, due to an insufficient number of answer 
sheets, also some universities in Anatolia were 
chosen. Since the hypotheses of the study depend 
on the comparison of (/) public universities 
and private universities and (it) opinions of 
administrators and academicians, the sample size 
was determined to meet minimum calculation 
requirements. Since a 12-item ipsative scale was 
used, the researchers aimed to reach a number of 
participants at least 15 times the number of items 
on the scale. Considering that some of the surveys 
distributed would not be returned, it was decided 
that a total of 200 surveys would be distributed, 
half of them given to administrators and the second 
half to academicians without any administrative 
responsibilities. In the same manner, half of the 
surveys were intended for public universities 
and the other half for private universities. The 
population and unit of measurement consists 
of instructors, all from universities in Turkey. 
Assistant professors were deemed to be a part of 
this group since they teach classes. More than 300 
surveys were sent to randomly selected universities 
with 176 being returned. Of these 176, five surveys 
were eliminated for the other reasons, rendering a 
total of 171 surveys forming the sample group. The 
survey return rate was 59%. 

Measurement Tool and Implementation 

Two structured survey forms were used in the study. 
The first form was developed for administrators and 
the second for academicians. Both survey forms are 
composed of the same content, save for two questions 
in the demographic section. Those questions not 
related to administrative roles and activities were 
deleted from the academician forms and some 
expressions related to performance indicators 
were changed to render the survey suitable for this 
group. The measurement tool consists of two parts. 
In the first part, demographic questions related to 
employer organizations, branches of science, and 
academic titles were asked to both administrators and 
academicians. The second part of the survey consists 
of a behavior-anchored rating scale. The characteristic 
of this tool is that it provides five different behavior 
options to respondents in relation to a certain 
performance indicator, asking which options best 
suits the respondent. The second measurement tool 
is “The Scale of Level of Participation to Performance 
Indicators.” This scale was developed based on both the 
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researchers’ experience in academic administration 
and also the literature review. It was concluded that it 
would be most beneficial to limit the items to twelve 
indicators. Both survey forms consisted of six pages 
and took about 30 minutes to complete. A part of 
the survey was conducted via face-to-face interviews 
and the remaining surveys were sent to academicians 
via email. The behavior-anchored scale is a type of 
a formative scale focusing on validity rather than 
reliability. As such, the validity analysis of the items 
was tested on a pilot study performed on a small group. 
As the scale used in the study is not a reflective scale, 
neither factor analysis nor specifying reliability were 
applied. It is known that in formative measurement 
models, statistical techniques are insufficient both 
for evaluation and for determining the reliability 
of manifest variables (Gudergan, Ringle, Wende, & 
Will, 2008). Since manifest variables in formative 
measurement scales constitute the structure, they are 
named as composite variables. The hidden conceptual 
structure is not disputed (Freeze & Raschke, 2007). 
Again, according to the same writers, the internal 
consistency between items (reliability), correlation 
between items, and error scores were not investigated 
(Freeze & Raschke, 2007, p. 1484). 

In the Behavior-Anchored Rating Scale, 5 levels 
were specified and each one of these levels was 
organized in a way that they could include the other. 
As such, a participant marking the fifth level means 
that this participant thinks it absolutely necessary 
to monitor performance indicators. A score of 1 
or 2 indicates that the respondent believes that the 
monitoring level should be at a minimum whereas 
a score of 4 or 5 indicates the opposite. 

Variables 

Dependent Variables: Each item in the 12-item 
behavior-anchored scale was determined as a 
dependent variable. When the aforementioned 
sub-dimensions are taken into consideration, it is 
seen that five items relate to teaching style, four 
items to the use of course materials, one item to 
student levels of success in the course, one item 
to measuring satisfaction levels, and one item to 
measuring attendance. These dimensions were 
taken as group variables. The measurement tool 
used was not an attitude measuring scale aiming to 
reveal a hidden structure, and accordingly, the rule 
of “at least three items” in determining a dimension 
could be disregarded. The twelve dependent 
variables were established to ascertain the 
following: (I) The attendance rate of academicians. 
This indicator aims to understand how many times 
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in a semester an academician did not attend class or 
sent an assistant instead. (2) Whether academicians 
started and ended class on time. This indicator aims 
to establish how frequently the academicians starts 
and ended class by means of three random checks 
to be performed in a single semester. Moreover, 
any related student complaints could be added 
to this indicator. (3) The courses presentation 
(teaching style). The aim of this indicator is for the 
academician to self-report regarding their ability 
to successfully implement the predetermined 
teaching methods and techniques. (4) Academician 
relations with their students. This indicator aims 
to obtain the department head’s assessment of the 
academician using a five-level scale. (5) The means 
used to enrich teaching methods. This indicator asks 
the academician to self-report as to the percentage 
of the audio-visual material actually used that 
had been planned for use at the beginning of the 
semester. (6) How strictly academicians followed 
to the syllabus. The aim of this indicator is three- 
dimensional. The first dimension is to keep up 
with planned timing, the second is to comply with 
the plan, and the third is the review and update of 
the plan (Kurz, Mueller, Gibbons, & DiCataldo, 
1989). (7) The use of course materials. Using all 
educational materials for which the academician 
held the students responsible for. (8) Academicians’ 
compliance with set theoretical lab hours. This 
indicator asks the academician to self-report as to 
what extent they complied with planned laboratory 
or exercise hours. (9) Academicians’ use of the course 
book. This indicator aims to determine the rate an 
academician used the recommended course book 
in measuring students’ learning. (10) Academicians’ 
attendance-taking performance. This indicator aims 
to determine how strictly the academician keeps 
attendance records of students in compliance with 
the stated policy. (11) Student satisfaction levels. This 
indicator aims to measure how satisfied students 
are with the course and provide feedback to the 
academician. (12) Success of the lessons taught; this 
indicator aims to have the academician distribute 
a form gauging class success and then consider the 
results while making assessments. 

Scale Levels: Each of the measurement items is 
evaluated using behavior and attitude expressions 
classified under five levels. The expression 
“academicians should be completely free” is coded 
as a “1” and the phrase “academicians should report 
to management under the established system and 
prepare statistical data” is coded as a “5.” The 
intermediate levels of 2 through 4 measure how 
strictly respondents believe that the indicators 
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should be monitored. The most relaxed view is to 
let academicians be completely free, to allow them 
to act under their own academic discretion. The 
most strict view supports the structured process, 
requiring academicians to regularly compile specific 
digital data to be reported to the administration and 
to prepare concrete, monitorable, and measurable 
data at the end of the semester. Between these views 
are ones which hold that academicians should be 
tracked; tracked and provided with explanations; or 
tracked, provided with explanations, and required 
to provide feedback to students. 

Independent Variables: 19 independent variables 
were determined for the administrator group and 
18 for the academicians. These are as follows: (1) 
the type of institution, (2) administrative position, 
(3) title, (4) administrative experience, (5) branch 
of science, (6) quality assurance committee, (7) 
age of the quality committee, (8) self-assessment 
applications and the age of such practices, (9) 
the performance of scales gauging satisfaction, 

(10) how frequently satisfaction scales are used, 

(11) satisfaction measurements, (12) feedback 
received from satisfaction scales, (13) keeping 
up with the syllabus, (14) compliance with the 
syllabus, (15) permissions for course materials 
to be copied, (16) yearly activity reports, (17) 
board minutes, (18) how often board meetings 
are held, and (19) personal self-assessment. On 
the one hand, the independent variables aim to 
obtain non-personal academic information about 
academic administrators and academicians, and 
on the other hand, they aim to establish whether 
or not quality works and performance indicator 
practices exist in their respective institutions. Since 
some of the independent variables were related to 
the dependent variables, whether there existed a 
relationship or not is what was examined. 

Research Questions and Hypotheses 

The independent variables were deemed as research 
questions and were also used as an instrument to 
test the hypothesis. The findings of the research 
questions have been summarized in a table. The 
aim of constructing such a summarized table is 
to ensure that a global assessment is being made 
about the independent variables. The following 
hypotheses whose aims are to draw conclusions and 
reach certain judgments are also tests in the study: 

1. The average scores of academicians differ from 
administrator scores in their understanding of 
EPIE components. 


2. Both the self-assessment and average scores 
under EPIE components differ from one another. 

The values belonging to the three independent 
variables were obtained in relation to the average 
scores of the three EPIE components. The reason 
for using the average scores of the components 
instead of the total score for the EPIE is that the 
correlation coefficient of the dimensions was 
found to be relatively low. Moreover, the behavior- 
anchored rating scale that was used is not regarded 
as a complete index. 

Analysis 

After the frequency distribution for an important 
part of the demographic data in the measurement 
tool had been found, central distribution statistical 
computations were performed for two variables 
with differing data structures. To see the distribution 
of the participants through behavior-anchored 
rating scale items, the benefits of the box graphic 
were used. In order to reduce the dimensional 
structure of the rating scale, exploratory factor 
analysis was performed. In order to compute 
the number of components, the eigen value and 
scree graph techniques were used. The purpose 
here is to establish which education performance 
indicators are not in accordance with the assumed 
dimensions but are in accordance with the results of 
the calculations. It is thus possible to test whether 
a relation exists between the average scores of the 
covered variables by arranging components and 
the demographic variables. Either the t-test, for 
two independent samples, or the single directional 
variance analysis was used to test the hypotheses 
depending on whether the demographic variables 
had two alternative or multi-alternative content. 

The reason for choosing the method of explanatory 
factor analysis was that no factor structure related 
to educational performance indicators had been 
conceptualized yet in the literature. Since the twelve 
items are grouped under three components, and 
since such components are related to each other at 
a medium level, the oblique rotation technique was 
applied and then each component was subjected 
to either a Cronbach alpha internal consistency 
analysis or a reliability analysis. A limited pilot 
test was performed to establish the validity of 
the scales items. This test required three research 
assistants to assess the educational performance 
indicator items via a five-level scale. Research 
assistants examined the scales surface, content, 
and internal consistency validity. Both inter-item 
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and item-total score correlations were performed 
to test internal consistency. The feedback from 
the assistants was considered and the required 
corrections were performed. The double per time 
correlation analyses that was performed showed the 
correlation coefficient between the arbitrators to be 
between .82 and .90 (p < .000). The Cronbach alpha 
reliability coefficient of the scale was determined to 
be .92 after assessment from the three arbitrators 
during the preliminary stage. 

Findings of the Study 
Findings from the Study Questions 

This section first provides the general results of 
the independent variables and then compares the 
results of the Educational Performance Indicators 
Scale between (i) administrators and academicians 
then (ii) private universities and public universities. 
Finally, the respective distribution graphics are 
presented. The final scale items of the educational 
performance indicators comprised of three 
components and nine items were determined by 
factor analysis. The nine items are known to have a 
high level of reliability. 

Box Graphics Depicting EPIE Monitoring 

It has always been wondered how administrators 
and academicians assess the implementation and 
monitoring of educational performance indicators 
as well as how they are distributed. This curiosity 
relates to whether there is a dispute or acceptance 
between these two groups exists, and if so, to what 
degree. Undoubtedly, it is best to have a high level of 
shared acceptance so as to increase the overall quality 
of the educational process. The respective analysis 
has been made using the distribution graphic, box 



graphic. The y-axis demonstrates approval levels 
toward EPIE, whereas the x-axis indicates any 
existing administrator function or university group. 
Each graphic handles one of the nine variables in the 
final scale, which were determined through factor 
analysis. These variables are as follows: academician 
attendance, timeliness in arriving to and leaving 
from class, course presentation, enrichment of 
teaching style, compliance with the course plan, 
course materials, basic course book, monitoring 
student attendance, monitoring student satisfaction, 
and monitoring course success. The first graphic 
provides the results according to management 
duties and the second graphic provides results 
according to university type. 

1. Academician Course Attendance: The opinions 
related to the monitoring of academician course 
attendance may differ. As the difference between 
the expectations of the directors and approaches 
of the academicians increases, the organizational 
structure and education standards are harmed. 
Figures 1 and 2 demonstrate the opinions of 
the participants as per the roles of director and 
academicians and on the basis of being a private 
university or a public university. 

If one interprets the box graphics, it can be seen that 
despite the same median values for the directors 
and academicians in Figure 1, 50% of academicians 
believe that the task is their own responsibility and 
expect lesser control exerted on them. Considering 
Figure 2, the median value in private universities 
is 2, whereas it is about 2.5 in public universities. 
The greater variance in public universities means 
that the opinions on “monitoring” are more widely 
distributed. Private universities, on the other hand, 
demonstrate a more regular opinion starting from 
a medium level of monitoring towards letting 
academicians be totally unmonitored. It may be 



Figure 1: Results of variable 1 for the academician and Figure 2: Results of variable 1 for the public and pnvate 
director groups. university groups. 
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Figure 3: Results of variable 2 for the academician and director groups. 

understood that the management approach of 
the private sector is more or less reflected in the 
behavior of the people. A portion of the people in 
public universities thinks that much more serious 
monitoring and control is necessary. 

2. Timeliness of Academicians in Arriving and 
Leaving: The second item in the fore plan, thanks 
to factor analysis, is related to the timeliness of 
academicians in arriving to and leaving from class. 
The respective results are given in Figures 3 and 4. 

When examining the figures, it may be seen that the 
approach of the directors and private universities 
are similar. The academicians demonstrated a wide 
range of opinions about monitoring their class 
arrival and departure routines. However, in general, 
more than 50% demonstrated opinions desiring 
more freedom on this item. This dominance most 
likely originates from the structural factors of 
public universities. 

3. Academician’s Level of Applying the Teaching 
Methods and Techniques: The teaching techniques 
item starts with distribution of the course programs 
to the students at the beginning of the semester 
and covers course conferences, an interactive 
evaluation, homework assignments, presentation 



Figure 4: Results of variable 2 for the public and private university groups. 

sessions, and mentoring or consultancy techniques. 
Monitoring the abilities of the academicians to 
apply such techniques and identify the levels 
in which the techniques should be applied will 
increase the frequency of use and number of 
techniques used. The responses of the participants 
are given in Figures 5 and 6. 

The median value is two for the academician and 
director group, meaning that the academicians 
do not want over-interference in their teaching 
methods and techniques. However, a significant 
number of the academicians provided opinions 
showing permission for the monitoring of their 
teaching styles. This approach may arise from 
a sort of dissatisfaction. The private and public 
universities group, in turn, do not demonstrate a 
serious difference. Both organizations deemed that 
providing “certain recommendations” for “teaching 
methods and techniques” would be sufficient. 
The real difference lies in the perceptions of the 
academicians and the expectations of the directors. 
The academicians expressed that in their opinion 
“a report is necessary at the end of the semester 
in relation to the level of implementation of the 
teaching methods and techniques.” 



Figure 5: Results of variable 3 for the academician and director groups. 


Figure 6: Results of variable 3 for the public and private university groups. 
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Figure 7; Results of variable 4 for the academician and director groups. 


4. Enriching the Course Teaching Style: Enriching 
the course teaching style means assigning homework 
to students; requiring presentations; making 
students use internet sources; the use of projection 
devices; and favoring surveys, photographs, and 
article evaluations as additional learning tools. The 
item teaching methods and techniques is closely 
related to enriching the course teaching style. The 
conference method, allowing student participation, 
and consulting with students are all approaches in 
the fore plan of teaching methods. However, the 
material used for strengthening the course becomes 
more important for enriching it. The results of the 
assessment are provided in Figures 7 and 8. 

The results resemble those for the item teaching 
methods and techniques. No serious difference is 
observed between private universities and public 
universities. However, academicians stated that this 
item should be their responsibility in most cases 
when compared to the opinions of the directors. 
Seventy-five percent of the academicians marked 
three or lower. 

5. Monitoring the Course Plan: Monitoring the 
academicians’ course plans (teaching plan) that 
teachers decided at the beginning of the semester 
and keeping up with such plans is another area 
deemed significant for educating students in a 
quality manner. The division heads may monitor 



Figure 9: Results of variable 5 for the academician and director groups. 
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Figure 8: Results of variable 4 for the public and private university groups. 


on several levels whether the academicians keep up 
with their teaching plans. In general, a wide range 
of practices exists, starting from solely monitoring 
until eventually warning the academician about 
the matter. The results under this educational 
performance indicator for the directors and 
academicians are provided in Figure 9, and for 
the private universities and public universities in 
Figure 10. 

Seventy-five percent of academicians have a median 
value lower than 2 and approximately 50% of the 
directors marked a median value in the same range. 
Thus, 25% of the academicians thought that this 
item should be handled more strictly. At least 50% 
of the academician group thought that semester 
course plans fall under their responsibility. Twenty- 
five percent, in turn, demonstrated a more open 
approach to managerial controls. A parallelism is 
present in the opinions of the academicians and 
public universities. The distribution of the opinions 
of the director generally indicates similarities with 
the opinion profile of private universities. Private 
universities give more importance to monitoring 
the course and teaching plans as well as making 
adjustments to the same when compared with 
public universities. 

6. Course Materials: This variable examined 
whether academicians prepare their own course 



Figure 10: Results of variable 5 for the public and private university groups. 
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Figure 11: Results of variable 6 for the academician and director groups 

books or notes in relation to their courses, the 
number of pages or sections from the book 
that they hold students responsible for, and the 
course materials that comply with the teaching 
(course) plan. Some problematic practices may be 
exemplified such as holding students responsible 
for a 600-page book, making students write 50 to 
60 pages of course notes throughout the semester, 
comprising exams based solely on class notes, and 
so forth. The manner of using course materials 
should be monitored in this way if the aim is to 
monitor educational performance. It is important 
that academicians make their own selections 
while keeping the educational knowledge load 
and intensity on the students close or equal to the 
omnipresent norms. Figures 11 and 12 provide the 
distribution graphics for both groups. 

The median value of academicians for the 
development and use of course materials and 
volume for which students will be responsible was 
found to be 1. The median value for directors is 2. 
This means that academicians perceive this area 
totally under their responsibility. When one looks at 
the director group, it can be seen that they resemble 
a wide range reaching the level of “recommending.” 



Figure 12: Results of variable 6 for the public and private university groups. 

Public universities also demonstrated a range 
reaching “statistics formation.” Fifty percent of 
academicians marked higher than the “general 
monitoring” item, whereas this percentage was only 
25% in the director group. This variable can then be 
understood as “underestimated” by the directors. 
Fifty percent of academicians desired total freedom, 
and the remaining 50% requested more control and 
monitoring, which consequently forms a spiral 
structure in which the components feed each other. 
When one compares private universities and public 
universities, close median values are observed 
(1.5 for public universities and 2.0 for private 
universities). The perception of “trust” may be felt 
in private universities, whereas public universities 
make a strong notion of the need for control and 
monitoring. At least 25% of the participants from 
public universities provided opinions in line with 
higher-level monitoring and control. 

7. Basic Course Book: It is deemed favorable for 
the education process that courses are taught in the 
framework of a well-structured and well-adjusted 
course book according to the level of the students, 
one that provides sufficient details and depth. 
Such a book should not necessarily include all 



Figure 14: Results of variable 7 for the public and private university groups. 


369 




Figure 13: Results of variable 7 for the academician and director groups 
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the subjects of the 14-week curriculum. It should, 
however, be a source work suitable for any time 
reference, extending and easing the context of 
learning for students. Monitoring the availability 
and competency of the basic course book and 
making use of other books, articles, research, 
and course notes are factors that will assure the 
quality of education as an educational performance 
indicator. The responsibilities related to the 
qualities of the basic course book and other course 
materials may be left totally to the division heads or 
academicians, or they may be monitored at certain 
levels. Observing such qualities means that this 
variable is a performance indicator. Figures 13 and 
14 provide the distribution graphics for the director 
and academician groups and the private and public 
university group. 

All the groups demonstrated a common attitude in 
relation to the item basic course book. This comes 
in the form of “showing general interest” and 
“checking if the basic course book has been picked 
or not” among 75% of the participants. Twenty-five 
percent of all groups provided opinions approving 
“holding meetings with academicians in relation to 
basic course books” and “making recommendations 
to academicians in such meetings as related to 
copyrights, number of pages for which students will 
be responsible, photocopies, etc.” These participants 
provided opinions that show a desire for advanced 
works to increase educational quality and improve 
the processes. 

8. Monitoring Educational Satisfaction: 

Monitoring the student level of satisfaction 
with their course and instructor is a frequently 
preferred method for improving educational 
processes. However, many universities do not 
have a well-thought-out, planned, systematic 
method for “measuring student satisfaction.” As 
satisfaction with the university, program, course, 
teaching method, and academician mix with 


each other, the results of such measurements do 
not provide the level of desired contribution for 
improvement of the processes. Under this variable, 
1 means “academician does not trust measurement 
results,” 2 means “general assessment,” 3 means 
“find quite valuable and worthy of importance,” 
4 “feedback about the academicians via a closed 
envelope, including a review of their behaviors,” 
and 5 “keeping related statistics and performance 
of periodical comparisons over many years.” The 
results are given in Figures 15 and 16. 

Fifty percent of the participants in all groups 
positioned themselves in the choices “general 
assessment” and “find quite valuable.” The 
percentage of the participants who “do not trust the 
measurement results” and “request feedback about 
the academicians via a closed envelope, including a 
review of their behaviors” were both at 25%. It may 
be understood that certain policies are required to 
turn the ideas of the 25% (deem such monitoring 
as unimportant) to a more positive array. It may be 
achieved by making the approach of measurement, 
the methodology of measurement, measurement 
tools, and analysis methods healthier. One should 
also not assess the position of the 50% group as 
a healthy placement. This position points out a 
procedural implementation. To improve education 
processes, an important portion of the participants 
should position themselves at 3 or higher. It 
would be favorable if the university directors stop 
perceiving satisfaction measurements as automatic 
processes, but on the contrary, as a significant work 
whose results will be assessed and discussed by the 
division or main-branch boards. Since no differing 
structure could be seen between public universities 
and private universities, it may be said that this 
implementation is realized using a similar approach 
in both organizations. 

9. Monitoring Student Success: Universities 
generally do not monitor what percentage of students 



Figure 15: Results of variable 8 for the academician and director groups. Figure 16: Results of variable 8 for the public and private university groups 
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Figure 17: Results of variable 9 for the academician and director groups. 

pass a course, fail, or repeat the same course for their 
second or third time. To stay a step ahead of this, 
one can examine the success rates and distribution 
of successful students under five levels. In a way, 
monitoring student success reveals the success of 
the academician. Designating most of the students 
as successful or unsuccessful, a very high or low 
general grade level may provide an idea about the 
measuring approach of the academician. If success 
reports are not monitored, then academicians may 
move out of a “reasonable” measurement standard. 
In relation to monitoring success, 1 means “I do not 
perform success monitoring or grade distribution 
assessment,” 2 means “I generally monitor success 
rates,” 3 means “I make a related distribution 
assessment at the end of the semester,” 4 means “I 
prepare a report on the success of the course at the 
end of the semester,” and 5 means “I monitor the 
statistical developments over years.” The results are 
given under Figures 17 and 18. 

A similar distribution is seen for all groups. The 
academicians, directors, private universities, and 
public universities provided a “general monitoring 
of the course success” with a median value of 2. 
Fifty percent of all participants from all groups 
provided their opinions as “generally monitoring or 
conducting a general assessment at the end of the 
semester.” Only 25% of the participants rose to the 
level of “I make a related distribution assessment 
at the end of the semester.” The success rate and 
success distribution of each course should be 
assessed by the division and main branch boards. 
However, in many universities the divisions and 
academic boards of the main branches do not 
perform work on the quality of education. 

Findings Related to Factor Analysis 

The oblique rotation technique as well as the method 
of basic components analysis were applied in this 
study on the 12 items of educational performance 


Figure IS: Results of variable 9 for the public and private university groups. 

indicators, aiming to reduce the number of 
indicators by means of establishing underlying 
dimensions or revealing unrelated indicators, 
thereby only including the related indicators with 
high representation in the final scale. A total of 
171 valid surveys were assessed. Five surveys were 
eliminated on several grounds. Maxwell (1959) and 
Tobias and Carlson (1969) recommended applying 
the Bartlett sphericity test before implementation 
of factor analysis (Dziuban & Shirkey, 1974). Thus, 
a chi-square value of 965.85, sd. 66, p = .000 < .05 
was obtained. The p value being lesser than .05 
means the H 0 hypothesis testing the unrelatedness 
of the variables in the population was rejected. The 
second stage applied the suitability test of Kaiser- 
Mayer-Olkin on the sample size. The test requires 
this size to be at least .60. The OKM test value 
was calculated at .854. This result proves that the 
sample size is sufficient for obtaining factors or for 
assessing the arisen factors. 

The Kaiser parameter is defined as 1 for finding 
factors, and the scree graphic is also used. The self¬ 
value of a factor (eigenvalue) shows the amount 
that the variance is explained by such a factor. Since 
one can assume that the indicators under the EPIE 
scale are related to each other, oblique rotation is 
preferred. The following results were obtained from 
the factor analysis results performed in line with 
the aforementioned explanations. 

According to the correlation matrix, it was seen 
that most of the variables are related to each other 
at .30 and higher. The communalities provide the 
partnership percentage of each variable with the 
factor. The values under the extraction title show 
the variability percentage of an item caused by the 
related factors. The communality value of the first 
variable was found to be .792, meaning that 79% 
of the variability of this item may be explained 
by the respective factor. The sub-limit for the 
communality percentage is .30. In other words, 
items having a communality rate less than .30 






















Educational Sciences: Theory & Practice 


Table 2 

The Component Loads of the Education Performance Indicators Scale 


Variables 


Component 1 

Component 2 

Component 3 

v7 

Course materials used by the academician 

.849 



v9 

Monitoring the Basic Course Book 

.812 



v5 

Enriching the Teaching Style 

.750 



v6 

Keeping in line with the Teaching Plan determined 
at the Beginning of the Semester 

.734 



vl 

Attendance of the Academician 


.888 


w2 

Academician Timeliness in Entering and Leaving the Class 


.887 


v3 

Teaching Style, Presentation, and Course Management 
of Academician 


.835 


vll 

Monitoring of Student Satisfaction 



.817 

vl2 

Monitoring the Course Success of Students 



.802 


should be favorably eliminated from the scale. The 
communality values of all 12 variables of the study 
were found to be higher than 0.30. Factor analysis 
showed three factors with self-values higher than 1. 
The total variance explanation percentage of these 
three factors, cumulatively, is 66%. In terms of total 
variance, the first factor explains 44.8%, the second 
factor, 11.9%, and the third factor, 9.2%. The pattern 
matrix and structure matrix tables were examined 
all together to establish which items would be in 
the final scale and which items would be taken 
out. Table 2 lists the correlation coefficients of the 
variables with the factors. The intersection point 
of the variables with the factor loads should not 
be determined independently, but in line with the 
frame of the figures from the literature. According 
to the literature, it is generally more logical to take 
items with factor loads higher than .60 or .70. Thus, 
it was decided to place 9 items in the final scale with 
component coefficients higher than .70. 


The factor represented by variables v7, v9, v5, and 
v6 from the educational performance indicators 
scale under the first component coverage is course 
material. Items vl, v2, and v3 are academician 
behavior, and indicators vll and vl2 are student 
behaviors. The reliability analysis of the nine-item 
scale was performed in the final stage, and the 
Cronbach alpha coefficient was .961 for the entire 
scale. Reliability analyses performed for the sub- 
dimensional coverage gave an alpha coefficient of 
.849 for the first component, .852 for the second 
component, and .587 for the third component. 
It can be said that the reliability coefficient of the 
third component was rather low. Confirmatory 
factor analysis was used to temporarily confirm 
the three-component structure, which arose 
among all three types of participants in the study. 
SPSS program AMOS 21 (Arbuckle, 2006) was 
used for this purpose. First, the chi-square test 
values are examined to understand the compliance 
of the three-factor model that arose from the 
confirmatory factor analysis of the EPIE Scale and 
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Figure 19: Three component EPIE indicators. 
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the data gathered from the field. If the sample size is 
sufficient in size, then a chi-square goodness-of-fit 
index value may be an important parameter to use 
in interpretation. The chi-square value of the study 
was 61.30 and the degree of freedom was 24 with a 
p-value of .000. The zero hypothesis for chi-square 
is determined to be the sample covariance matrix 
data obtained from the measurement data equal to 
the population covariance matrix data, and since 
p was less than .05, it was concluded that the data 
of the covariance matrices were different. Thus, it 
was concluded that the three-component model 
did not comply with the data. A significant non- 
compliance was present between the determined 
three-factor model and the observed covariance 
matrix (See Figure 19). 

The researchers also regarded the results of other 
types of chi-square statistical analyses to assess the 
model and data compliance. One of these was the 
CMIN test in the AMOS program. The CMIN was 
found to be 61.0, sd of 24; p = 00; and CEMIN/DF: 
2.55. It was also observed that the model did not 
comply under the CMIN calculation. Specialists have 
stated that a “good” compliance may be mentioned 
for a CEMIN/DF value of 1, an “acceptable” 
compliance for values between 1 and 2, and a 
“sometimes” compliance for values between 1 and 3 
or 1 and 5 (Karplus, 2013). The RMSE Value, which 
is one of the most informative indexes of structural 
equality models, was found to be .10. This value also 
resembles a non-compliance, or “weak compliance.” 

Findings Related to the Hypothesis Tests 

The hypothesis tests were calculated by taking the 
average scores from the ( a ) course material, ( b ) 
academician behavior, and (c) student behavior 
dimensions obtained after factor analysis. The first 
hypothesis was tested between the academicians 
and director groups. Thus, in the first instance, 
equal distribution of the values was examined 
using Levene’s equality of variances test, and it was 
understood that the data met this prerequisite (F = 
7.882; p = .006). f-test analysis was conducted for 
the first dimension of EPIE, which is monitoring 
the use of course materials, and it was understood 
that no statistically significant difference existed 
between the average grades of the academicians and 
the directors (t = .186, sd. 169; p = .853; MD: -.258; 
MDCI: -.300 - .248). The confidence interval range 
of the difference between the averages was found 
to be quite narrow. Non-existence of a statistically 
significant difference does not mean such a 
difference does not exist scientifically or in real 


life. The Levene test analysis made on monitoring 
academician behaviors proved that the variances of 
both groups are equal (F = 17.605; p = .000). f-test 
analysis is conducted for the item monitoring the 
use of course materials, and it was understood that 
no statistically significant difference exists between 
the average grades of the academicians and the 
directors (f = 1.245, sd: 169; p = .215; MD: -.217; 
MDCI: .562 - .127). The equality of the variances 
between the groups’ test conducted for the final 
dimension student behavior provided negative 
results (F = .005; p = .945). Thus, the calculation 
values realized under the assumption of differing 
variances were considered, f-test analysis was 
conducted for monitoring student behaviors and 
it was understood that no statistically significant 
difference existed between the average grades of the 
academicians and the directors (f = 1.453, sd: 169; p 
= .148; MD: .178; MDCI: .420 - .063). 

The second hypothesis was tested for the public 
and private universities. Comparisons were 
made between the average points of the three 
components, which are the sub-dimensions of 
the EPIE ipsative scale. For the first dimension, 
the equality of variance between the groups’ test 
provided positive results ( F = .425; p = .044). f-test 
analysis was conducted on monitoring the use of 
course materials, and it was understood that no 
statistically significant difference existed between 
the average grades of the public and private 
universities (f = .270, sd: 169; p = .787; MD: .037; 
MDCI: .312 - .237). The equality of the variances 
between the groups’ tests gave negative results for 
the second dimension (F = .039; p = .844), and thus 
the calculation values realized under the assumption 
of differing variances were considered to apply. 
T-test analysis was conducted for the dimension 
monitoring the behaviors of academicians and it 
was understood that no statistically significant 
difference existed between the average grades of 
public and private universities (f =.114, sd: 169; p = 
.909; MD: .020; MDCI: .367 - .327). The equality of 
the variances between the groups’ tests gave negative 
results for the third dimension (F = .714; p = .399), 
and thus the calculation values realized under the 
assumption of differing variances were considered 
applicable, f-test analysis was conducted for the 
dimension monitoring the behaviors of students and 
it was understood that no statistically significant 
difference existed between the average grades of the 
public and private universities (f = .028, sd: 169; p = 
.977; MD: .003; MDCI: .246 - .239). 
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The third hypothesis was mentioned as “no 
difference between the self-assessment scores and 
the average score of the EPIE components.” The 
purpose of this hypothesis is to detect if the EPIE 
is affected by objective factors, such as director or 
academician, private university or public university, 
or by psychological factors such as personality and 
self-assessment. Equality of variance between the 
groups’ tests was checked and it was seen that the 
groups had the same variance (F = .983; p = .000). 
According to single-direction variance analysis 
results, it was understood that the average points 
showing the dimension monitoring the use of course 
materials might differ according to the type of self- 
assessment. A statistically significant difference 
existed between the self-assessment points and 
the average points of the EPIE components (F = 
11.681; sd: 4; p = .000). Also, a statistically significant 
relation was detected between the self-assessment 
points and the dimension monitoring the behaviors 
of the academician. Equality of variance between 
the groups test was F = 5.522; p = .000 whereas the 
variance analysis values were F = 6.966; sd: 4 ;p = .000. 
The Levene test was performed as a prerequisite for 
self-assessment points and the dimension monitoring 
student behaviors, and obtained the results F = .347 
and p = .846. The single direction variance analysis 
test result was not considered for this group since the 
group variances were not equal. 

Discussion 

Priority was given to the hypothesis tests while 
evaluating the findings. A significant relationship 
could not be established between the point averages 
of the EPIE components of the director-academician 
and private university-public university groups, 
which may be assumed as “objective phenomenon.” 
The differences between the groups in relation to the 
levels of monitoring the performance indicators did 
not provide statistically significant results. However, 
one may also not claim that any significant difference 
exists scientifically or in real life. Score averages tend 
to deviate from reality in Likert-type scales as well 
as in ipsative scales. Low points may be leveled with 
high points and a significant relationship cannot be 
established in the case of middle-degree intensification. 
The sample size of 171 individuals might have been 
insufficient. However, a statistically significant 
relationship was additionally detected between the 
self-assessments of the participants, which may be 
deemed as a psychological phenomenon, and the 
score averages of two of the EPIE components. These 
two components are the dimensions monitoring the 


use of course materials and monitoring the behaviors 
of the academician. The group variances were not 
equal for the third component, and the results of the 
variance analysis were accordingly not taken into 
consideration. It was understood that the participants 
deemed themselves mostly as persons with attitudes of 
“permitting freedom” and “being easy going,” and this 
affected the scores more than the EPIE components. 
This proves that the attitudes and approaches of people 
while establishing the educational performance 
indicators affected their self-designations and self- 
assessment moods rather than their official positions. 
Explanatory factor analysis was used to examine the 
factorial infrastructure of the 12 theoretical EPIE, 
and a three-component structure was obtained. 
’’Insufficiency of analysis” was the second of three 
components with only two indicators. If the sample 
size was increased or the number of indicators were 
higher, the number of items in the third component 
might increase. This study did not aim to develop an 
attitude scale. The aim was to establish a lesser but 
more effective number of educational performance 
indicators to serve as an improvement of educational 
quality by means of academicians and directors. The 
more academicians and directors assess higher points 
for the necessity of monitoring an indicator, the more 
probable it will be that such an indicator will be taken 
under the monitoring plan and become more effective. 
As a beginning, this study shows nine indicators that 
may be used for monitoring educational performance 
with academicians taking a role in the education 
process. After the establishment of the factorial 
structure, confirmatory factor analysis was performed 
in the frame of the structural equality model to see to 
what extent the three-component model that arose 
complied with the gathered data. This analysis has 
a temporary quality. In fact, examining the three- 
component model with data from another sample 
drawn from the same population would provide more 
substantial results. Such a study would be the factual 
examination of the theoretical information. The 
model-data compliance could not be seen from the 
confirmatory factor analysis for which the chi-square 
tests and other compliance tests were conducted. 
Perhaps more advanced analyses are required which 
consider the independence or dependence of the 
error factors. Some authors prefer large sample sizes 
and others small volume samples for compliance. 
However, the appropriate representation of the 
measurement universe by the determined indicators 
is as important as the size of the sample. Thus, it would 
be favorable to perform focus group research on the 
12 indicators of the EOIE index. 
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Cabrera et al. (2001) stated that educational 
performance during university education is 
a mutual result of previously existing student 
characteristics as well as the education provided 
by the university during this time. They expressed 
that student characteristics prior to attending a 
university include talents, social-economic status 
or background, educational expectations, ethnicity, 
and gender factors. Indicators during university 
education include in-class and out-of-class 
activities. In-class activities were examined under 
the variables of teaching and lecturing methods, 
curriculum, relationships between students, and 
relationships with faculty members. Out-of-class 
activities were approached as social activities 
and student club activities. In this research on 
educational performance, indicators were evaluated 
only within the framework of the classroom. In the 
survey form from Cabrera et al., they came up with 
5 factors from 26 experiences that specifically were 
aimed at measuring in-class experiences, and they 
found that these five factors demonstrated 62% 
of the total variance. Three out of the five factors 
were related to training activities and the other two 
were related to class environment, which included 
tolerance toward women and minorities. In their 
study, three in-class factors that could be related 
are as follows: (a) interaction and feedback in the 
classroom environment, ( b ) cooperative learning, 
and (c) teachers subject knowledge and careful 
organization of the subject. These variables are 
indirectly related to the variables used in our study, 
such as course material, course plan, and interactive 
teaching. It is seen that Cabrera et al. dealt with 
and evaluated educational performance indicators 
not only in-class but also in a wider framework. 
In various studies, educational performance 
indicators were studied at national, institutional, 
and campus levels, and reports were produced 
(Chalmers, 2008). Indicators at the departmental 
levels were classified under the subtitles of input, 
process, and results indicators. Indicators discussed 


in this research were focused on process and 
results. Because of this, results obtained from this 
research cannot be compared with other research 
at similar levels. Furthermore, in many research 
studies, performance indicators were associated 
with “program outputs.” Providing program output 
is seen as having equal value as to the substantiation 
of performance indicators (Felder, 2003). This 
means that specific educational indicators directed 
at programs rather than general educational 
indicators came into prominence. Performance 
indicators in this research were discussed from the 
perspectives of “class pedagogy” and “education 
standards.” There are researchers like Kozma 
(2014), who have directed their attention to this 
subject in the literature. However, there is still no 
consensus among researchers about which basic 
indicators must constitute educational indicators. 

Our final assessment relates to the distribution 
variance of the three-component and nine- 
indicator EPIE index with regard to the 
academicians and directors group and the public 
university and private university group. If general 
evaluation were the case, it could be said that the 
score distribution of directors resembles the scores 
of the private universities, and the scores of the 
academicians resemble the scores of the public 
universities. Most of the academicians expect 
the implementation of educational performance 
indicators at a a minimal level, whereas directors 
and private universities provided opinions between 
the 2 and 3 point intervals in support of monitoring 
practices. However, one is aware that quality 
directors of universities expect EPIE monitoring 
levels to be much higher, perhaps at the 3 to 4 point 
level. As EPIE monitoring expectations relate to 
personal self-assessment more than the managerial 
position of the person, it seems that any work on 
creating self-awareness, informing, and providing 
sensibility would be effective in raising the EPIE 
monitoring levels to higher scores. 
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